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Abstract 

We consider the two-fold problem of rep- 
resenting collective beliefs and aggregating 
these beliefs. We propose modular, transitive 
relations for collective beliefs. They allow 
us to represent conflicting opinions and they 
have a clear semantics. We compare them 
with the quasi-transitive relations often used 
in Social Choice. Then, we describe a way 
to construct the belief state of an agent in- 
formed by a set of sources of varying degrees 
of reliability. This construction circumvents 
Arrow's Impossibility Theorem in a satisfac- 
tory manner. Finally, we give a simple set- 
theory-based operator for combining the in- 
formation of multiple agents. Wc show that 
this operator satisfies the desirable invariants 
of idempotence, commutativity, and associa- 
tivity, and, thus, is well-behaved when iter- 
ated, and we describe a computationally ef- 
fective way of computing the resulting belief 
state. 

Keywords: representation of beliefs, multi-agent sys- 
tems 

1 Introduction 

We are interested in the multi-agent setting where 
agents are informed by sources of varying levels of reli- 
ability, and where agents can iteratively combine their 
belief states. This setting introduces three problems: 
(1) Finding an appropriate representation for collec- 
tive beliefs; (2) Constructing an agent's belief state by 
aggregating the information from informant sources, 
accounting for the relative reliability of these sources; 
and, (3) Combining the information of multiple agents 
in a manner that is well-behaved under iteration. 
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The Social Choice community has dealt extensively 
with the first problem (although in the context of 
representing collective preferences rather than beliefs) 
(cf. (Sen 1986)). The classical approach has been to 
use quasi-transitive relations (of which total pre-orders 
are a special subclass) over the set of possible worlds. 
However, these relations do not distinguish between 
group indifference and group conflict, and this distinc- 
tion can be crucial. Consider, for example, a situa- 
tion in which all members of a group are indifferent 
between movie a and movie b. If some passerby ex- 
presses a preference for a, the group may very well 
choose to adopt this opinion for the group and borrow 
a. However, if the group was already divided over the 
relative merits of a and b, we would be wise to hesitate 
before choosing one over the other just because a new 
supporter of a appears on the scene. We propose a 
representation in which the distinction is explicit. We 
also argue that our representation solves some of the 
unpleasant semantical problems suffered by the earlier 
approach. 

The second problem addresses how an agent should 
actually go about combining the information received 
from a set of sources to create a belief state. Such a 
mechanism should favor the opinions held by more re- 
liable sources, yet allow less reliable sources to voice 
opinions when higher ranked sources have no opin- 
ion. True, under some circumstances it would not 
be advisable for an opinion from a less reliable source 
to override the agnosticism of a more reliable source, 
but often it is better to accept these opinions as de- 
fault assumptions until better information is available. 
(Maynard-Reid II and Shoham 2000) provides a solu- 
tion to this problem when belief states are represented 
as total pre-orders, but runs into Arrow's Impossibil- 
ity Theorem (Arrow 1963) when there are sources of 
equal reliability. As we shall see, the generalized rep- 
resentation allows us to circumvent this limitation. 

To motivate the third problem, consider the follow- 



ing dynamic scenario: A robot controlling a ship in 
space receives from a number of communication cen- 
ters on Earth information about the status of its en- 
vironment and tasks. Each center receives informa- 
tion from a group of sources of varying credibility or 
accuracy (e.g., nearby satellites and experts) and ag- 
gregates it. Timeliness of decision-making in space is 
often crucial, so we do not want the robot to have to 
wait while each center sends its information to some 
central location for it to be first combined before be- 
ing forwarded to the robot. Instead, each center sends 
its aggregated information directly to the robot. Not 
only does this scheme reduce dead time, it also allows 
for "anytime" behavior on the robot's part: the robot 
incorporates new information as it arrives and makes 
the best decisions it can with whatever information it 
has at any given point. This distributed approach is 
also more robust since the degradation in performance 
is much more graceful should information from indi- 
vidual centers get lost or delayed. 

In such a scenario, the robot needs a mechanism for 
combining or fusing the belief states of multiple agents 
potentially arriving at different times. Moreover, the 
belief state output by the mechanism should be invari- 
ant with respect to the order of agent arrivals. We will 
describe such a mechanism. 

The paper is organized as follows: After some pre- 
liminary definitions and a discussion of the approach 
to aggregation taken in classical Social Choice, we in- 
troduce modular, transitive relations for representing 
generalized belief states. We then describe how to 
construct the belief state of an agent given the be- 
lief states of its informant sources when these sources 
are totally pre-ordered. Finally, we describe a simple 
set-theory-based operator for fusing agent belief states 
that satisfies the desirable invariants of idempotence, 
commutativity, and associativity, and we describe a 
computationally effective way of computing this belief 
state. 



1. reflexive iff x < x for a; G O. It is irreflexive iff 

X ^ X for X 

2. symmetric iffx<y^y<x for x,y G^. It is 
asymmetric iffx<y=>y^x for x,y Gfl. It 
is anti-symmetric iffx<yAy<x=^x = y for 
x,y gSI. 

3. the strict version of a relation <' over Cl iff 

x<y<^x<'y/\y-^'x for x,y G il. 

4- total iff X < yV y < X for x, j/ G f2. 

5. modular iffx<y=f-x<zyz<y for x,y,z £ Q. 

6. transitive iff x<yAy<z=^x<z for 

x,y,z e rj. 

7. quasi-transitive iff its strict version is transitive. 

8. the transitive closure of a relation <' 
over fl iff x < y <^ 3wo, ■ ■ ■ ,Wn G fl. 
X = wq <'■••<' Wn = y for some integer n, 
for x,y 

9. acyclic iff Vwq, ■ ■ ■ ,Wn G voq < ■ ■ ■ < Wn im- 
plies Wn -ft Wo for all integers n, where < is the 
strict version of <. 

10. a total pre-order iff it is total and transitive. It is 
a total order iff it is also anti-symmetric. 

11. an equivalence relation iff it is reflexive, symmet- 
ric, and transitive. 

Proposition 1 

1. The transitive closure of a modular relation is 
modular. 

2. Every transitive relation is quasi-transitive. 

3. (Sen 1986) Every quasi-transitive relation is 
acyclic. 



2 Preliminaries 

We begin by defining various well-known properties of 
binary relations^; they will be useful to us throughout 
the paper. 

Definition 1 Suppose < is a relation over a finite 
set f2, i.e., <C Q x We shall use x <y to denote 
{x,y) E< andx ^y to denote {x,y) ^<. The relation 
< is: 



Given a relation over a set of alternatives and a subset 
of these alternatives, we often want to pick the subset's 
"best" elements with respect to the relation. We define 
this set of "best" elements to be the subset's choice set: 

Definition 2 If < is a relation over a finite set < 
is its strict version, and X CCl, then the choice set of 
X with respect to < is 

C{X, <) = {x€X -.^x' G X. x' < x}. 



^We only use binary relations in this paper, so we will 
refer to them simply as relations. 



A choice function is one which assigns to every subset 
X a non-empty subset of X: 



Definition 3 A choice function over a finite set 17 is 
a function / : 2" \ 2" \ such that f{X) C X for 
every X Cfl. 

Now, every acyclic relation defines a choice function, 
one which assigns to each subset its choice set: 

Proposition 2 (Sen 1986) Given a relation < over a 
finite set Q, the choice set operation C defines a choice 
function iff < is acyclic? 

If a relation is not acyclic, elements involved in a cycle 
are said to be in a conflict because we cannot order 
them: 

Definition 4 Given a relation < over a finite 
set Vt, X and y are in a conflict wrt < iff 
there exist wq, . . . ,Wn, zo, ■ ■ ■ , Zm 0, such that 
X = Wo < ■ ■ ■ < Wn = y = zq < ■ ■ ■ < Zm = X, whcrc 
x,y gSI. 

3 Aggregation in Social Choice 

We are interested in belief aggregation, but the com- 
munity historically most interested in aggregation has 
been that of Social Choice theory. The aggregation is 
over preferences rather than beliefs, so the discussion 
in this subsection will focus on representing prefer- 
ences; however, as we shall see, the results are equally 
relevant to representing beliefs. In the Social Choice 
community, the standard representation of an agent's 
preferences is a total prc-ordcr. Each total pre-order 
:<i is interpreted as describing the weak preferences of 
an individual i, so that x <iy means i considers al- 
ternative X to be at least as preferable as alternative 
y.^ li X y and y :<i x, then i is indifferent between 
X and y. 

Unfortunately, Arrow's Impossibility Theorem (Arrow 

1963) showed that no aggregation operator over total 
pre-orders exists satisfying the following small set of 
desirable properties: 

Definition 5 Let f be an aggregation operator over 
the preferences ^i, ^„ of n individuals, respec- 
tively, over a finite set of alternatives il, and let 
^ =/(^i,-..,d«). 

• Restricted Range; The range of f is the set of 

total pre-orders over n. 

^Sen's uses a slightly stronger definition of choice sets, 
but the theorem still holds in our more general case. 

^The direction of the relation symbol is unintuitive, but 
standard practice in the belief revision community. 



• Unrestricted Domain; The domain of f is the set 
of n-tuples of total pre-orders over fl. 

• Pareto Principle; If x -<i y for all i, then x -< y. 

• Independence of Irrelevant Alternatives (IIA); 
Suppose ^' = f{<[, . . . , ^;). //, for x,y£n, 
X diiV iff X ^[y for all i, then x <y iff x :<' y. 

• Non-Dictatorship; There is no individual i such 
that, for every tuple in the domain of f and every 
x,y £ Cl, X -<i y implies x -<y. 

Proposition 3 (Arrow 1963) There is no aggregation 
operator that satisfies restricted range, unrestricted do- 
main, (weak) Pareto principle, independendence of ir- 
relevant alternatives, and nondictatorship. 

This impossibility theorem led researchers to look for 
weakenings to Arrow's framework that would circum- 
vent the resiilt. One was to weaken the restricted range 
condition, requiring that the result of an aggregation 
only satisfy totality and quasi-transitivity rather than 
the full transitivity of a total pre-order. This weak- 
ening was sufficient to guarantee the existence of an 
aggregation function satisfying the other conditions, 
while still producing relations that defined clioic;c func- 
tions (Sen 1986). However, this solution was not with- 
out its own problems. 

First, total, quasi-transitive relations have unsatisfac- 
tory semantics. If < is total and quasi-transitive but 
not a total pre-order, its indifference relation is not 
transitive: 

Proposition 4 Let :< be a relation over a finite set 
fl and let ~ be its symmetric restriction (i.e., x ^ y 
iff X ^ y and y ^ x). If ^ is total and quasi-transitive 
but not transitive, then ^ is not transitive. 

There has been much discussion as to whether or not 
indifference should be transitive; in many cases one 
feels indifference should be transitive. If Deb enjoys 
plums and mangoes equally and also enjoys mangoes 
and peaches equally, we would conclude that she also 
enjoys plums and peaches equally. It seems that total 
quasi-transitive relations that are not total pre-orders 
cannot be understood easily as preference or indiffer- 
ence. 

Since the existence of a choice function is generally suf- 
ficient for classical Social Choice problems, this issue 
was at least ignorable. However, in iterated aggrega- 
tion, the result of the aggregation must not only be us- 
able for making decisions, but must be interpretable as 
a new preference relation that may be involved in later 



aggregations; consequently, it must maintain clean se- 
mantics. 

Secondly, the totality assumption is excessively restric- 
tive for representing aggregate preferences. In general, 
a binary relation ^ can express four possible relation- 
ships between a pair of alternatives a and b: a <h and 
b a, b <a and a y^b, a <b and b <a, and a ^b 
and b ^ a. Totality reduces this set to the first three 
which, under the interpretation of relations as repre- 
senting weak preference, correspond to the two strict 
ordcrings of a and 6, and indifference. However, con- 
sider the situation where a couple is trying to choose 
between an Italian and an Indian restaurant, but one 
strictly prefers Italian food to Indian food, whereas the 
second strictly prefers Indian to Italian. The couple's 
opinions are in conflict, a situation that does not fit 
into any of the three remaining categories. Thus, the 
totality assumption is essentially an assumption that 
conflicts do not exist. This, one may argue, is appro- 
priate if we want to represent preferences of one agent 
(but see (Kahneman and Tversky 1979) for persua- 
sive arguments that individuals are often ambivalent). 
However, the assumption is inappropriate if we want to 
represent aggregate preferences since individuals will 
almost certainly have differences of opinion. 

4 Generalized Belief States 

Let us turn to the domain of belief aggregation. A 

total pre-order over the set of possible worlds is a 
fairly well-accepted representation for a belief state in 
the belief revision community (Grove 1988; Katsuno 
and Mendelzon 1991; Lehmann and Magidor 1992; 
Gardenfors and Makinson 1994). Instead of prefer- 
ence, relations represent relative likelihood, instead of 
indifference, equal likelihood. For the remainder of the 
paper, assume we are given some language C with a 
satisfaction relation ^ for C Let W be a finite, non- 
empty set of possible worlds (interpretations) over C. 
Suppose ^ is a total pre-order on W. The belief re- 
vision literature maintains that the conditional belief 
"if p then q'' (where p and q are sentences in C) holds 
if all the worlds in the choice set of those satisfying p 
also satisfy q; we write Bel{plq). The individual's un- 
conditional beliefs are all those where p is the sentence 
true. If neither the belief plq nor its negation hold in 
the belief state, it is said to be agnostic with respect 
to plq, written Agn{plq). 

It should come as no surprise that belief aggregation 
is formally similar to preference aggregation and, as a 
result, is also susceptible to the problems described in 
the previous section. We propose a solution to these 
problems which generalizes the total pre-order repre- 



sentation so as to capture information about conflicts. 

4.1 Modular, transitive states 

We take strict likelihood as primitive. Since strict like- 
lihood is not necessarily total, it is possible to repre- 
sent agnosticism and conflicting opinions in the same 
structure. This choice deviates from that of most au- 
thors, but are similar to those of Kreps (Kreps 1990, 
p. 19) who is interested in representing both indiffer- 
ence and incomparability. Unlike Kreps, rather than 
use an asymmetric relation to represent strict likeli- 
hood (e.g., the strict version of a weak likelihood rela- 
tion), we impose the less restrictive condition of mod- 
ularity. 

We formally define generalized belief states: 

Definition 6 A generalized belief state -< is a mod- 
ular, transitive relation over W. The set of possible 
generalized belief states over W is denoted B. 

We interpret a ~< b to mean "there is reason to con- 
sider a as strictly more likely than 6." We represent 
equal likelihood, which we also refer to as "agnosti- 
cism," with the relationship defined such that x y 
if and only \i x y and y x. We define the confiict 
relation corresponding to ^, denoted oo, so that xooy 
\'&x <y and y ^ x. It describes situations where there 
are reasons to consider either of a pair of worlds as 
strictly more likely than the other. In fact, one can 
easily check that cx) precisely represents conflicts in a 
belief state in the sense of Definition 4. 

For convenience, we will refer to generalized belief 
states simply as belief states for the remainder of the 
paper except when to do so would cause confusion. 

4.2 Discussion 

Let us consider why our choice of representation is 

justified. First, we agree with the Socnal Clioicx; com- 
munity that strict likelihood should be transitive. 

As we discussed in the previous section, there is of- 
ten no compelling reason why agnosticism/indifference 
should not be transitive^; we also adopt this view. 
However, transitivity of strict likelihood by itself does 
not guarantee transitivity of agnosticism. A sim- 
ple example is the following: ^={(a, c)}, so that 
~= {(o, 6), (6, c)}. However, if we buy that strict like- 
lihood should be transitive, then agnosticism is transi- 
tive identically when strict likelihood is also modular: 

Proposition 5 Suppose a relation ~< is transitive and 
~ is the corresponding agnosticism relation. Then ^ 
is transitive iff -< is modular. 



In summary, transitivity and modularity arc necessary 
if strict likelihood and agnosticism are both required 
to be transitive. 

We should point out that conflicts are also transitive 
in our framework. At first glance, this may appear 
undesirable: it is entirely possible for a group to dis- 
agree on the relative likelihood of worlds a and b, and 
h and c, yet agree that a is more likely than c. How- 
ever, we note that this transitivity follows from the 
cycle-based definition of conflicts (Definition 4), not 
from our belief state representation. It highlights the 
fact that we are not only concerned with conflicts that 
arise from simple disagreements over pairs of alterna- 
tives, but those that can be inferred from a series of 
inconsistent opinions as well. 

Now, to argue that modular, transitive relations are 
sufScient to capture relative likelihood, agnosticism, 
and conflicts among a group of information sources, 
we first point out that adding irreflexivity would give 
us the class of relations that are strict versions of total 
prc-ordcrs, i.e., conflict-free. Let T be the set of total 
pre-orders over W, 7^, the set of their strict versions. 

Proposition 6 The set of irreflexive relations in B is 
isomorphic to T and, in fact, equals 7^ . 

Secondly, the following representation theorem shows 
that each belief state partitions the possible worlds 
into sets of worlds either all equally likely or all poten- 
tially involved in a conflict, and totally orders these 
sets; worlds in distinct sets have the same relation to 
each other as do the sets. 

Proposition 7 -<G B iff there is a partition 
W = (Wo, . . . , W„> ofW such that: 

1. For every x ^Wi and y G Wj, i ^ j implies i < j 
iffx-<y. 

2. Every Wi is either fully connected (w ~<w' for all 
w,w' GWi) or fully disconnected (w -fi w' for all 
w,w' e Wi). 

Figure 1 shows three examples of belief states: one 
which is a total prc-order. one which is the strict ver- 
sion of a total pre-order, and one which is neither. 

Thus, generalized belief states are not a big change 
from the strict versions of total pre-orders. They 
merely generalize these by weakening the assumption 
that sets of worlds not strictly ordered are equally 
likely, allowing for the possibility of conflicts. Now 
we can distinguish between agnostic and conflicting 
conditional beliefs. A belief state -< is agnostic about 




(a) (b) (c) 



Figure 1: Three examples of generalized belief states: 
(a) a total pre-order, (b) the strict version of a total 
pre-order, (c) neither. (Each circle represents all the 
worlds in W which satisfy the sentence inside. An arc 
between circles indicates that w -< w' for every w in 
the head circle and w' in the tail circle; no arc indi- 
cates that w T^w' for each of these pairs. In particular, 
the set of worlds represented by a circle is fully con- 
nected if there is an arc from the circle to itself, fully 
disconnected otherwise.) 

conditional belief p?g (i.e., Agn{plq)) if the choice set 
of worlds satisfying p contains both worlds which sat- 
isfy q and -^q and is fully disconnected. It is in conflict 
about this belief, written Con{plq), if the choice set is 
fully connected. 

Finally, we compare the representational power of our 
definitions to those discussed in the previous section. 
First, B subsumes the class of total pre-orders: 

Proposition S T C B and is the set of reflexive rela- 
tions in B. 

Secondly, B neither subsumes nor is subsumed by the 
set of total, quasi-transitive relations, and the inter- 
section of the two classes is T. Let Q be the set of 
total, quasi-transitive relations over W, and Q<, the 
set of their strict versions. 

Proposition 9 

1. QnB^T. 

2. B%Q. 

3. Q^BifW has at least three elements. 
4- QcBifW has one or two elements. 

Because modular, transitive relations represent strict 
preferences, it is probably fairer to compare them to 
the class of strict versions of total, quasi-transitive re- 
lations. Again, neither class subsumes the other, but 
this time the intersection is : 

Proposition 10 

1. Q^nB = T<. 



2. B%Q<. 

3. Q< %BifW has at least three elements. 
4- Q< CBifW has one or two elements. 

In the next section, we define a natural aggregation 
policy based on this new representation that admits 
clear semantics and obeys appropriately modified ver- 
sions of Arrow's conditions. 

5 Single-agent belief state 
construction 

Suppose an ag(^nt is informed by a set of sources, each 
with its individual belief state. Suppose further that 
the agent has ranked the sources by level of credibility. 
We propose an operator for constructing the agent's 
belief state -< by aggregating the belief states of the 
sources in S while accounting for the credibility rank- 
ing of the sources. 

Example 1 We will use a running example from our 

space robot domain to help provide intuition for our 
definitions. The robot sends to earth a stream of 
telemetry data gathered by the spacecraft, as long as 
it receives positive feedback that the data is being re- 
ceived. At some point it loses contact with the auto- 
matic feedback system, so it sends a request for infor- 
mation to an agent on earth to find out if the failure 
was caused by a failure of the feedback system or by an 
overload of the data retrieval system. In the former 
case, it would continue to send, data, in the latter, de- 
sist. As it so happens, there has been no overload, but 
the computer running the feedback system has hung. 
The agent consults the following three experts, aggre- 
gates their beliefs, and sends the results back to the 
robot: 

1. Sp, the computer programmer that developed the 
feedback program, believes nothing could ever go 
wrong with her code, so there must have been an 
overload problem. However, she admits that if 
her program had crashed, the problem could rip- 
ple through to cause an overload. 

2. Sm, the manager for the telemetry division, unfor- 
tunately has out- dated information that the feed- 
back system is working. She was also told by the 
engineer who sold her the system that overloading 
could never happen. She has no idea what would 
happen if there was an overload or the feedback 
system crashed. 



3. St, the technician working on the feedback sys- 
tem, knows that the feedback system crashed, but 
doesn't know whether there was a data-overload. 
Not being familiar with the retrieval system,, she is 
also unable to speculate whether the data retrieval 
system would have overloaded if the feedback sys- 
tem had not failed. 

Let F and D be propositional variables representing 
that the feedback and data retrieval systems, respec- 
tively, are okay. The belief states for the three sources 
are shown in Figure 2. 




Figure 2: The belief states of Sp, Sm, and st in Exam- 
ple 1. 

Let us begin the formal development by defining 
sources: 

Definition 7 S is a finite set o/ sources. With each 
source s € S is associated a belief state B. 

We denote the agnosticism and conflict relations of a 
source s by w"* and ixi*, respectively. It is possible to 
assume that the belief state of a source is conflict free, 
i.e., acyclic. However, this is not necessary if we allow 
sources to suffer from the human malady of "being 
torn between possibilities." 

We assume that the agent's credibility ranking over 
the sources is a total pre-order: 

Definition 8 TZ is a totally ordered finite set o/ ranks. 

Definition 9 rank : S ~^TZ assigns to each source a 
rank. 

Definition 10 □ is the total pre-order over S in- 
duced by the ordering over TZ. That is, s □ s' iff 
rank{s) > rank{s'); we say s' is as credible as s. 3s 
is the restriction of^ to S C S. 



We use □ and = to denote the asyninicitrie and sym- 
metric restrictions of respectively.^ The finiteness 
of <S (7^) ensures that a maximal source (rank) al- 
ways exists, which is necessary for some of our results. 
Weaker assumptions are possible, but at the price of 
unnecessarily complicating the discussion. 

We are ready to consider the source aggregation prob- 
lem. In the following, assume an agent is informed by 
a set of sources S C S. We look at two special cases — 
equal-ranked and strictly-ranked source aggregation — 
before considering the general case. 

5.1 Equal-ranked sources aggregation 

Suppose all the sources have the same rank so that 
3s is fully connected. Intuitively, we want take all 
offered opinions seriously, so we take the union of the 

relations: 

Definition 11 If S C S, then Un{S) is the relation 

u 



By simply taking the union of the source belief states, 
we may lose transitivity. However, we do not lose mod- 
ularity: 

Proposition 11 If S C S, then Un{S) is modular but 
not necessarily transitive. 

Thus, we know from Proposition 1 that we need only 
take the transitive closure of Un{S) to get a belief 
state: 

Definition 12 If S Q S, then AGRUn{S) is the re- 
lation Un{S)~^. 

Proposition 12 If S C S, then AGRUn{S) e B. 

Not surprisingly, by taking all opinions of all sources 
seriously, we may generate many conflicts, manifested 
as fully connected subsets of W. 

Example 2 Suppose all three sources in the space 
robot scenario of Example 1 are considered equally 
credible, then the aggregate belief state will be the fully 
connected relation indicating that there are conflicts 
over every belief. 

5.2 Strictly-ranked sources aggregation 

Next, consider the case where the sources are strictly 
ranked, i.e., 3s is a total order. We define an operator 



such that lower-ranked sources refine the bc;lief states 
of higher ranked sources. That is, in determining the 
ordering of a pair of worlds, the opinions of higher- 
ranked sources generally override those of lower-ranked 
sources, and lower-ranked sources are consulted when 
higher-ranked sources are agnostic: 

Definition 13 If S C S, then AGRRf{S) is the re- 
lation 

|(a;,y) :3sgS.x y A (Vs' □ s G 5. a; Ri**' 

The definition of the AGRRf operator does not rely 
on ^5 being a total order, and we will use it in this 
more general setting in the following sub-section. How- 
ever, in the case that 3s is a total order, the result of 
applying AGRRf is guaranteed to be a belief state. 

Proposition 13 If S C S and 3s o, total order, 
then AGRRf {S) eB. 

Example 3 Suppose, in the space robot scenario of 
Example 1, the technician is considered more cred- 
ible than the manager who, in turn, is considered 
more credible than the programmer. The aggregate be- 
lief state, shown in Figure 3, informs the robot cor- 
rectly that the feedback system has crashed, but that it 
shouldn't worry about an overload problem and should 
keep sending data. 




''Note that, unlike the relations representing belief 
states, > and □ are read in the intuitive way, that is, 
"greater" corresponds to "better." 



Figure 3: The belief state after aggregation in Exam- 
ple 3 when S( □ Sm □ Sp. 

Note that this case of strictly-ranked sources is al- 
most exactly that considered in (Maynard-Reid II and 
Shoham 2000), except that the authors are not able to 
allow for conflicts in belief states. A surprising result 
they show is that standard AGM belief revision (Al- 
chourron et al. 1985) can be modeled as the aggrega- 
tion of two sources, the informant and the informee, 
where the informant is considered more credible than 
the informee. 



5.3 General aggregation 

In the general case, we may have several ranks rep- 
resented and multiple sources of each rank. It will 
be instructive to first consider the following seem- 
ingly natural strawman operator, AGR*: First com- 
bine cqui-rank sources using AGRUn, then aggregate 
the strictly-ranked results using what is essentially 
AGRRf: 

Definition 14 Let S C S. For any r ^ TZ, 
let <r= AGRUn{{s G S : rank{s) = r}) and w^', 
the corresponding agnosticism, relation. Also, let 
rank{S) = {r eUiBs e S. rank{s) = r}. AGR*{S) 
is the relation 

( , . BreTZ.xKrVA 1 
\ y> • (Vr' > r e ranks{S). x ^r' v) j 

AGR* indeed defines a legitimate belief state: 

Proposition 14 If S Q S, then AGR*{S) G B. 

Unfortunately, a problem with this "divide-and- 
conquer" approach is it assumes the result of ag- 
gregation is independent of potential interactions be- 
tween the individual sources of different ranks. Con- 
sequently, opinions that will eventually get overridden 
may still have an indirect effect on the final aggrega- 
tion result by introducing superfluous opinions during 
the intermediate equi-rank aggregation step, as the fol- 
lowing example shows: 

Example 4 Let >V={a, 6, c}. Suppose S <^ S 
such that S = {so, 81,82} with belief states 
{(6,a),(&,c)} and <''=<''^= {{a,b), {c,b)}, 
and where S2 □ si = Sq. Then AGR*{S) is 
{ (a, 6) , (c, 6) , (a, c) , (c, a),{a,a),{b,b),{c,c)}. All 
sources are agnostic over a and c, yet (a, c) and (c, a) 
are in the result because of the transitive closure in 
the lower rank involving opinions ({b,c) and {b,a)) 
which actually get overridden in the final result. 

Because of these undesired effects, we propose another 
aggregation operator which circumvents this problem 
by applying refinement (as defined in Definition 13) 
to the set of source belief states before infering new 
opinions via closure: 

Definition 15 T/ie rank-based aggregation 0/ a set 0/ 
sources S C S is AGR{S) = AGRRf {S)+. 

Encouragingly, AGR outputs a valid belief state: 

Proposition 15 7/5 C S, then AGR{S) G B. 



Example 5 Suppose, in the space robot scenario of 
Example 1, the technician is still considered more cred- 
ible than the manager and the programmer, but the 
latter two are considered equally credible. The aggre- 
gate belief state, shown in Figure 5, still gives the robot 
the correct information about the state of the system. 
The robot also learns for future reference that there 
is some disagreement over whether or not there would 
have been a data overload if the feedback system were 
working. 




Figure 4: The belief state after aggregation in Exam- 
ple 5 when St □ = Sp. 

We observe that AGR, when applied to the set of 
sources in Example 4, does indeed bypass the problem 
described above of extraneous opinion introduction: 

Example 6 Assume W, S , and 3 o-Te as in Exam- 
ple 4. AGR{S) = {{a,b),{c,b)}. 

We also observe that AGR behaves well in the special 
cases we've considered, reducing to AGRUn when all 
sources have equal rank, and to AGRRf when the 
sources are totally ranked: 

Proposition 16 Suppose S C S. 

1- If^s «s fully connected, AGRiS) = AGRUn{S). 
2. If^s is a total order, AGR{S) = AGRRf (S). 

5.4 Arrow, revisited 

Finally, a strong argument in favor of AGR is that 
it satisfies appropriate modifications of Arrow's condi- 
tions. Let / be an operator which aggregates the belief 
states <'*^ , • • • , over W of n sources si, . . . , s„ e 5, 
respectively, and let -< = f{<^^ , . . . ,<^"). We con- 
sider each condition separately. 

Restricted range The output of the aggregation 
function will be a modular, transitive belief state 
rather than a total pre-order. 



Definition 16 (modified) Restricted Range; The 
range of f is B. 

Unrestricted domain Similarly, the input to the 
aggregation function will be modular, transitive belief 
states of sources rather than total pre-orders. 

Definition 17 (modified) Unrestricted Domain; For 
each i, can be any member of B. 

Pareto principle Generalized belief states already 
represent strict likelihood. Consequently, we use the 
actual input and output relations of the aggregation 
function in place of their strict versions to define the 
Pareto principle. Obviously, because we allow for the 
introduction of conflicts, AGR will not satisfy the orig- 
inal formal Pareto principle which essentially states 
that if all sources have an unconflicted belief that one 
world is strictly more likely than another, this must 
also be true of the aggregated belief state. Neither 
condition is necessarily stronger than the other. 

Definition 18 (modified) Pareto Principle; // 
X y for all i, then x -<y. 

Independence of irrelevant alternatives Con- 
flicts are defined in terms of cycles, not necessarily 
binary. By allowing the existence of conflicts, we ef- 
fectively have made it possible for outside worlds to 
affect the relation between a pair of worlds, viz., by 
involving them in a cycle. As a result, we need to 
weaken IIA to say that the relation between worlds 
should be independent of other worlds unless these 
other worlds put them in conflict. 

Definition 19 (modifled) Independence of Irrelevant 
Alternatives (HA); Suppose s'^, . . . , e S such that 
Si = 5- for all i, and -<'= /(<^'i, . . . , <""). //, for 
x,y € W, X y iffx <■'» y for alii, x<^y, andx(^'y, 
then X ~<y iff x <' y. 

Non-dictatorship As with the Pareto principle def- 
inition, we use the actual input and output relations 
to define non-dictatorship since belief states repre- 
sent strict likelihood. From this perspective, our set- 
ting requires that informant sources of the highest 
rank be "dictators" in the sense considered by Ar- 
row. However, the setting originally considered by Ar- 
row was one where all individuals are ranked equally. 
Thus, we make this explicit in our new definition of 
non-dictatorship by adding the pre-condition that all 
sources be of equal rank. Now, AGR treats a set of 
equi-rank sources equally by taking all their opinions 
seriously, at the price of introducing confiicts. So, in- 



tuitively, there are no dictators. However, because Ar- 
row did not account for confiicts in his formulation, all 
the sources will be "dictators" by his definition. We 
need to modify the definition of non-dictatorship to 
say that no source can always push opinions through 
without them ever being contested. 

Definition 20 (modified) Non-Dictatorship; // 
Si = Sj for all i,j, then there is no i such that, for 

every combination of source belief states and every 
x,y G W, X y and y fi"' x implies x ~< y and 
y T^x. 

We now show that AGR indeed satisfies these condi- 
tions: 

Proposition 17 Let S = {si, . . . , s„} C S and 
AGRf{<^\. ..,<«")= AGR{S). AGRf satisfies (the 
modified versions of) restricted range, unrestricted 
domain, Pareto principle, IIA, and non- dictatorship. 

6 Multi-agent fusion 

So far, we have only considered the case where a sin- 
gle agent must construct or update her belief state 
once informed by a set of sources. Multi-agent fu- 
sion is the process of aggregating the belief states of a 

set of agents, each with its respective set of informant 
sources. We proceed to formalize this setting. 

An agent A is informed by a set of sources S C S. 
Agent A's induced belief state is the belief state 
formed by aggregating the belief states of its infor- 
mant sources, i.e., AGR{S). Assume the set of agents 
to fuse agree upon rank (and, consequently, 3).^ We 
define the fusion of this set to be an agent informed 
by the combination of informant sources: 

Definition 21 Let A = {Ai, . . . , An} be a set of 

agents such that each agent Ai is informed by Si C S. 
The fusion of A, written (V) {A), is an agent informed 

Not surprisingly given its set-theoretic definition, fu- 
sion is idempotent, commutative, and associative. 
These properties guarantee the invariance required in 
multi-agent belief aggregation applications such as our 
space robot domain. 

^We could easily extend the framework to allow for indi- 
vidual rankings, but we felt that the small gain in general- 
ity would not justify the additional complexity and loss of 
perspicuity. Similarly, we could consider each agent as hav- 
ing a credibility ordering only over its informant sources. 
However, it is unclear how, for example, crediblity order- 
ings over disjoint sets of sources should be combined into a 
new credibility ordering since their union will not be total. 



In the multi-agent space robot scenario described in 
Section 1, we only have a direct need for the belief 
states that result from fusion. We are only interested 
in the belief states of the original sources in as far as 
we want the fused belief state to reflect its informant 
history. An obvious question is whether it is possi- 
ble to compute the belief state induced by the agents' 
fusion solely from their initial belief states, that is, 
without having to reference the belief states of their 
informant sources. This is highly desirable because of 
the expense of storing — or, as in the case of our space 
robot example, transmitting — all source belief states; 
we would like to represent each agent's knowledge as 
compactly as possible. 

In fact, we can do this if all sources have equal rank. 
We simply take the transitive closure of the union of 
the agents' belief states: 

Proposition 18 Let A and S be as in Definition 21, 
-<^^ , agent Ai 's induced belief state, and Zjs,. fully con- 
nected. If A = @ (A), then {{JA^eA ^^")^ ^'s in- 
duced belief state. 

Unfortunately, the equal rank case is special. If we 
have sources of different ranks, we generally cannot 
compute the induced belief state after fusion using 
only the agent belief states before fusion, as the fol- 
lowing simple example demonstrates: 

Example 7 Let W = {a,b}. Suppose two agents Ai 

and A2 are informed by sources si with belief state 
{(a, 6)} and S2 with belief state <*^= {(6, a)}, 
respectively. Ai 's belief state is the same as si 's and 
A2 's is the same as 82 's. If si H S2, then the belief 
state induced by @ (^1, A2) is whereas if S2 □ si, 
then it is <^^. Thus, just knowing the belief states 
of the fused agents is not sufficient for computing the 
induced belief state. We need more information about 
the original sources. 

However, if sources are totally pre-ordered by credi- 
bility, we can still do much better than storing all the 
original sources. It is enough to store for each opinion 
of AGRRf{S) the rank of the highest-ranked source 
supporting it. We define pedigreed belief states which 
enrich belief states with this additional information: 

Definition 22 Let A be an agent informed by a set 
of sources S C S. A's pedigreed belief state is a pair 
{^,1) where -<= AGRRf{S) and I :^^ U such that 
l{{x,y)) = max{ranfc(s) : x y.s G 5}. We use 
to denote the restriction of A's pedigreed belief state 
to r, that is, -<r= Ki^^v)) = 

We verify that a pair's label is, in fact, the rank of 



the source used to determine the pair's membership in 
AGRRf{S), not that of some higher-ranked source: 

Proposition 19 Let A be an agent informed by a set 
of sources S Q S and with pedigreed belief state {^,1). 
Then 

X^rV 

iff 

3s € S. X <^ y Ar = rank{s)A 
(Vs' □ s e 5. a; . 

The belief state induced by a pedigreed belief state 
(^, I) is, obviously, the transitive closure of -<. 

Now, given only the pedigreed belief states of a set of 
agents, we can compute the new pedigreed belief state 
after fusion. We simply combine the labeled opinions 
using our refinement techniques. 

Proposition 20 Let A and S be as in Definition 21, 
3s, a total pre-order, and A = @ {A). If 

1. < is the relation 

( 3A, eA^ren.x yA | 

y) ■■ (^j^, >r&n.x j 

over W, 

2. I -.^—fTl such that 
l{{x,y)) = max{r : x -<^^ y,Ai G A}, and 

then {-<,l) is A's pedigreed belief state. 

Prom the perspective of the induced belief states, 
we are essentially discarding unlabeled opinions (i.e., 
those derived by the closure operation) before fusion. 
Intuitively, we are learning new information so we may 
need to retract some of our inferred opinions. After 
fusion, we re-apply closure to complete the new be- 
lief state. Interestingly, in the special case where the 
sources are strictly-ranked, the closure is unnecessary: 

Proposition 21 If A and S are as in Definition 21, 
□ s is a total order, and {~<,l) is the pedigreed belief 
state of® (A), then ^+=-<. 

Example 8 Let's look once m,ore at the space robot 
scenario considered in Example 1. Suppose the arro- 
gant programmer is not part of the telemetry team, but 
instead works for a company on the other side of the 
country. Then the robot has to request information 



from two separate agents, one to query the m,anager 
and technician and one to query the programmer. As- 
sume that the agents and the robot all rank the sources 
the same, assigning the technician rank 2 and the other 
two agents rank 1, which induces the same credibility 
ordering used in Example 5. The agents ' pedigreed be- 
lief states and the result of their fusion are shown in 
Figure 5. 
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Figure 5: The pedigreed belief states of agent Ai in- 
formed by Sm and st and of agent A2 informed by Sp, 
and the result of their fusion in Example 8. 

The first agent does not provide any information about 

overloading and the second agent provides incorrect in- 
formation. However, we see that after fusing the two, 
the robot has a belief state that is identical to what it 
computed in Exam,ple 5 when there was only one agent 
informed by all three sources ( we 've only separated the 
top set of worlds so as to show the labeling). Conse- 
quently, it now knows the correct state of the system. 
And, satisfyingly, the final result does not depend on 
the order in which the robot receives the agents' re- 
ports. 

The savings obtained in required storage space by 
this scheme can be substantial. Whereas explicitly 
storing all of an agent's informant sources S requires 
0(||5||2^) amount of space in the worst case (when 
all the sources' belief states arc fully connected rela- 
tions), storing a pedigreed belief state only requires 
C(2^) space in the worst case. Moreover, not only 
docs the enriched representation allow us to conserve 
space, but it also provides for potential savings in the 
efficiency of computing fusion since, for each pair of 
worlds, we only need to consider the opinions of the 
agents rather than those of all the sources in the com- 
bined set of informants. 

Incidentally, if we had used AGR* as the basis for 
our general aggregation, simply storing the rank of the 
maximum supporting sources would not give us suffi- 



cient information to compute the induced belief state 
after fusion. To demonstrate this, we give an example 
where two pairs of sources induce the same annotated 
agent belief states, yet yield different belief states after 
fusion: 

Example 9 Let W, S, and 3 be as in Exam- 
ple 4- Suppose agents Ai, A2, A[, and A'2 
are informed by sets of sources Si, S2, S[, and 
S'2, respectively, where Si = S2 = {S2}, S[ = {30,82}, 
and S'2 = {si, .S2}. AGR* dictates that the pedi- 
greed belief states of all four agents equal <''^^ with 
all opinions annotated with rank{s2). In spite 
of this indistinguishability, if A = @ ({Ai , A2}) and 
A' = ® {{A[,A'2}), then A's induced belief state 
equals K""^ , i.e., {(a, 6), (c, 6)}, whereas A' 's is 
{(a, 6), (c, 6), (a, c), (c, a), (a, a), (6, 6), (c, c)}. 



7 Conclusion 

We have described a semantically clean representa- 
tion for aggregate beliefs which allows us to represent 
conflicting opinions without sacrificing the ability to 
make decisions. We have proposed an intuitive oper- 
ator which takes advantage of this representation so 
that an agent can combine the belief states of a set 
of informant sources totally pre-ordered by credibility. 
Finally, we have described a mechanism for fusing the 
belief states of different agents which iterates well. 

The aggregation methods we have discussed here are 
just special cases of a more general framework based on 
voting. That is, we account not only for the ranking of 
the sources supporting or disagreeing with an opinion 
(i.e., the quality of support), but also the percentage of 
sources in each camp (the quantity of support). Such 
an extension allows for a much more refined approach 
to aggregation, one much closer to what humans of- 
ten use in practice. Exploring this richer space is the 
subject of further research. 

Another problem which deserves further study is de- 
veloping a fuller understanding of the properties of the 
Bel, Agn, and Con operators and how they interrelate. 
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Proposition 1 

1. The transitive closure of a modular relation is 
modular. 

2. Every transitive relation is quasi-transitive. 

3. (Sen 1986) Every quasi-transitive relation is 
acyclic. 

Proof: 

1. Suppose a relation < over finite set is modu- 
lar, and is the transitive closure of <. Sup- 
pose x,y,z&Q. and x <^ y. Then there ex- 
ist wq, . . . , Wn such that x — wq < ■ ■ ■ < Wn = y. 
Since < is modular and wq <Wi, either wo < z 
ov z < wi. In the former case, x = wq < z, so 
X <+ z. In the latter case, z < wi < ■ ■ ■ Wn = y, 
so z <+ y. 

2. Suppose f7 is a finite set, x,y, z G fl, < is a. tran- 
sitive relation over Q, and < is its strict version. 
Suppose X < y and y < z. Then x < y, y ^ x, 
y < z, and z ^ y. x < y and y < z imply x < z, 
and y < z and y ^ x imply z ^ x, both by tran- 
sitivity. So X < z. 



Proposition 2 (Sen 1986) Given a relation < over a 
finite set fl, the choice set operation C defines a choice 
function iff < is acyclic. 

Proof: See (Sen 1986). ■ 

Proposition 3 (Arrow 1963) There is no aggregaMon 
operator that satisfies restricted range, unrestricted do- 
main, (weak) Pareto principle, independendence of ir- 
relevant alternatives, and nondictatorship. 

Proof: See (Arrow 1963). ■ 

Proposition 4 Let :< be a relation over a finite set 
Q and let ^ be its symmetric restriction (i.e., x^y 
iff X ^ y and y ^ x). If ^ is total and quasi-transitive 
but not transitive, then ~ is not transitive. 

Proof: Let ^ be a total, quasi-transitive, non- 
transitive relation. First, such a relation exits: if 
n = {a, b, c}, it is easily verified that the relation 
Q X \ {(6, a)} is total, quasi- transitive, but not tran- 
sitive. 



Suppose X :< y and y ^ z but x z. By totality, 
z <x^ so z ^ X. \i X then z <y \yj quasi- 

transitivity, a contradiction. Thus, x ~ y. Similarly, 
if y ^ z, then y -< x, a contradiction, so y z. But 
z ^ X, so X z. Therefore, ~ is not transitive. ■ 

Proposition 5 Suppose a relation -< is transitive and 
~ is the corresponding agnosticism relation. Then ~ 
is transitive iff -< is modular. 

Proof: Suppose ~ is transitive and suppose x ^ z, 
x,y,z G W. We prove by contradiction: Suppose 

X y and y z. By transitivity, z y and y x, 
so X ~ 2/ and y ^ z. By assumption, x ~ z, so x 7^ z, 
a contradiction. 

Suppose, instead, -< is modular and suppose x ~ y 

and y ^ z, x, y, z G W. Then x y^ y, y y^ x, y y^ z, and 
z y^ y. By modularity, x y^ z and z 7^ x, so x ~ z. ■ 

Proposition 6 The set of irreflexive relations in B is 
isomorphic to T and, in fact, equals . 

Proof: Let x,y,z e W. Suppose B is irreflexive. 
Let ^ be defined as x <y iff y y^ x. We first show 
that -< is the strict version of <. Suppose <' is the 
strict version of If x -<' y, then x ^ y and y ^ x, 
so X -< y. If, instead, x -< y, then y 2^ x. By totality, 
a; ^ y, so X -<' y. 

We show that <<eT. li x y^y then y <x. Other- 
wise, X -< y. But since -< is irreflexive, y 7^ x (oth- 
erwise X ^ X by transitivity), so x ^ y and < is total. 
Next, suppose x ^ y and y ^ z. Then y yi. x and z yi. y. 
By modularity, z 7^ x, so x ^ z, so ^ is transitive. 

Now suppose T and -< is its strict version. First we 
show that -< is modular. Suppose x ~<y. Then x <y 
and y 2^ X. Since ^ is total, x ^ z or z ^ x. Suppose 
X < z. Whether y^z or y2<z,z^x by transitivity. 
Suppose, instead, z <x. Then z ^ y and y ^ z, both 
by transitivity. We conclude that x < z and z ;^ x, or 
z <y and y 2^ z, so x ^ z or z -< y. Second, transitiv- 
ity of ^ follows immediately from Proposition 1 and 
the transitivity of <. Finally, -< is irreflexive since it 
is asymmetric. ■ 

Proposition 7 B iff there is a partition 
W = {Wo, . . . ,Wn) ofW such that: 

1. For every x £Wi and y gWj, i^ j implies i < j 
iffx<y. 

2. Every Wi is either fully connected, (w ^ tv' for all 
Wyw' &Wi) or fully disconnected (w y^ w' for all 
w,w' e Wi). 



Proof: We refer to the conditions in the proposition 
as conditions 1 and 2, respectively. We prove each 
direction of the proposition separately. 

(=») Suppose -<€. B, that is, -< is a modular and tran- 
sitive relation over W. We use a series of definitions 
and lemmas to show that a partition of W exists satis- 
fying conditions 1 and 2. We first define an equivalence 
relation by which we will partition W. Two elements 
will be equivalent if they "look the same" from the 
perspective of every element of W: 

Definition 23 x = y iff for every z G W, x ^ z iff 
y -< z and z -< x iff z -< y. 

Lemma 7.1 = is an equivalence relation over W. 

Proof: Suppose x G W. For every z G W, x z iff 
x -< z and z -< x iff z ^ x, so x = x. Therefore, = is 
reflexive. 

Suppose x,y gW and x = y. Then for every z G W, 
X z iff y ^ z and z ^ x iff z y. But then for every 
z G W, y -< z iS x -< z and z -< y iff z -< x. Therefore, 
y = X, so = is symmetric. 

Suppose x,y,z G W, x = y, and y = z. Suppose fur- 
ther that w G W. By definition of =, x ^ w iff y ^ w 
and w ^ X iS w ^ y, and y -< w iS z -< w and w -< y 
iff w ^ z. Therefore, x ~< w iff z ~< w and -< x iff 
w -< z. Since w is arbitrary, x = z, so = is transitive. 
■ 

= partitions W into its equivalence classes. We use [w] 
to denote the equivalence class containing w, that is, 
the set {w' € W : w = w'}. Observe that two worlds 
in conflict always appear in the same equivalence class: 

Lemma 7.2 If x,y GW and xooy, then [x] = [y]. 

Proof: Suppose x, y G W and xooy. Since [x] is an 
equivalence class, it suffices to show that y G [x] , that 
is, X = y. Suppose z G W. By transitivity, if x -< z, 
then y -< z; if y -< z, then x -< z; if z -< x, then z -< y; 
and, if z -< y then z -< x. Thus, x ^ z iff y ^ z and 
z -< X iff z -< y, and since z is arbitrary, x = y. ■ 

We now define a total order over these equivalence 
classes: 

Definition 24 For all x, y G W, [x] < [y] iff [x] = [y] 
or X -<y. 

Lemma 7.3 < is well-defined, that is, if x = x' and 

y = y' , then x -i y iff x' ~< y' , for all x, x' . y, y' G W. 

Proof: Suppose x = x' and y = y' , x, x', y, y' G W. 
By the definition of =, for every z G W, x ^ z iff 



x' < z. In particular, x <y iff x' ^ y. Also by the 
definition of =, for every z' G W, z' -<y iff z' ~< y' . 
In particular, a;' -< y iff a;' -< y' . Therefore, a; -< y iff 
x' -< y'. ■ 

Lemma 7.4 < is a total order over the equivalence 
classes ofW defined by =. 

Proof: Suppose x,y,z G W. We first show that < 

is total. By definition of <, if x ~< y or y ~< x, then 
[x] < [y] or [y] < [x], respectively. Suppose x -/^ y and 
y X, and suppose z G W. By modularity of -<, 
X < z implies y < z, y ^ z implies x < z, z < x implies 
z ~< y, and z ^ y implies z ^ x, so x = y. Therefore, 
[a;] = [y], so [x] < [y] by the definition of <. 

Next, we show that < is anti-symmetric. Suppose 

[x] < [y] and [y] < [x]. Then [x] =^ [y] or x ~< y and 
y ^ X. In the former case we are done, in the latter, 
the result follows from Lemma 7.2. 

Finally, we show that < is transitive. Suppose [a;] < [y] 
and [y] < [z]. Obviously, if [x] = [y] or [y] = [x], then 
[x] < [z]. Suppose not. Then x ^ y and y ^ z, so 
x -< z hy the transitivity of -<. Therefore, [a;] < [y] 
by the definition of <. ■ 

We name the members of the partition Wq, . . . , W„ 

such that Wi < Wj iff i < j, where n is an integer. 
Such a naming exists since every finite, totally ordered 
set is isomorphic to some finite prefix of the integers. 

We now check that this partition satisfies the two 

conditions. For the first condition, suppose x € Wi, 
y GWj, and i ^ j. We want to show that i < j iff 
x ~<y. Since j, [x]^ [y\. Suppose i < j. Then 
i < j, so [x] < [y]. Since [x] =/= [y], x -< y by the def- 
inition of <. Now suppose, instead, that x ^ y. 
Then [x] < [y] by the definition of <, so i < j. Since 
[x] 7^ [y], y -A X by Lemma 7.2. Since [x] ^ [y] and 
y ^ X, [x] ^ [y] by the definition of <, so j ^ Thus, 
i < j. 

Finally, we show that each Wj is either fully connected 
or fully disconnected. Suppose x,y,z € Wi so that 
X = y = z. It suffices to show that x ~< x iS y ~< z. By 
the definition of =, a; -< a; iff y ^ a;, and x -< x iS x -< z. 
Suppose X < X. Then, y < x and a; ^ z, so y -< z by 
transitivity of -<. Suppose now, x -f^x. Then, y -/^ x 
and a; 7^ z, so y 7^ z by modularity of -<. 

{<=) Suppose W = (Wo, . . . , Wn) is a partition of W 

and ^ is a relation over W satisfying the given condi- 
tions. We want to show that -< is modular and tran- 
sitive. We first give the following lemma: 

Lemma 7.5 Suppose W is a partition of W and 



^ is a relation over W satisfying condition 1. If 
Wi, Wj e W, x e Wi, y e Wj, and x ^ y, then i < j. 

Proof: If i = j, we're done. Suppose i ^ j. Then, 
since a; ^ y, i < j by condition 1. ■ 

We now show ~< is modular. Suppose x &Wi, y £ Wj, 
and X -<y. Then i < j by Lemma 7.5. Suppose 
z e Wfe. Then i <k ov k < j hy the modularity of <. 
Suppose i < k OT k < j. Then x ^ z or z ^ y hy con- 
dition 1. Otherwise i = k = j, so x,y,z G Wi. Since 
X -<y, Wi is fully connected by condition 2, so x -< z 
(and z -< y). 

Finally, we show that -< is transitive. Suppose x G Wi, 
y GWj, z G Wk, X ^ y, and y ^ z. By Lemma 7.5, 
i < j and j < k, so i < k by the transitivity of <. 
Suppose i < k. Then x -< z by condition 1. Other- 
wise i = k = j, so x,y, z G Wi. Since x ~< y,Wi is fully 
connected by condition 2, so a; -< z. ■ 

(END OF PROPOSITION 7 PROOF) 

Proposition 8 T c B and is the set of reflexive rela- 
tions in B. 

Proof: We first show that T CB. Let T and 
x,y,z G W. By definition, < is transitive. Suppose 
X ^y. Since ^ is total, x ^ z or z ^ x. If 2; ^ a;, 
then z ^ y by transitivity, so ^ is modular. On the 
other hand, the empty relation over W is modular and 
transitive, but not total and, consequently, not in T. 

Now we show that -<€ B is in T if and only if it is 
reflexive. If <gT, it is total, so it is reflexive. If, 
instead, -< is reflexive, then x ^ x so, by modularity, 
x < y or y -< X. Thus, -< is total. And, since -<gB, it 
is transitive. ■ 

Proposition 9 

1. QnB^T. 

2. B%Q. 

3. Q%BifW has at least three elements. 
4- QcBifW has one or two elements. 

Proof: 

1. Suppose <G QO B. Then ^ is total and transitive 
and, hence, in T. Suppose T. By deflnition, 
^ is total. Also by definition, it is transitive, so 
by Proposition 1, it is quasi-transitive and, thus, 
in Q. By Proposition 8, ^G B and, so, in QdB. 



2. The empty relation is modular and transitive, but 
not total and, so, not in Q. 

3. Suppose a and b are distinct elements of W. The 
relation W x W \ {(6, a)} is total, and, since the 
strict version is {{a. b)} which is transitive, it is 
also quasi-transitive. However, if there are at least 
three elements in W, it is not transitive and, so, 
not in B. 

4. Suppose W has one element. Then B contains 
both possible relations over W, whereas Q con- 
tains only the fully connected relation over W. 

Suppose W has two elements a and b. Then B 

contains empty relation, the fully connected rela- 
tion, and all the remaining eight relations which 
contain either (a, b) or {b, a), but not both. Q, on 
the other hand, only contains the three reflexive 
relations containing either (a, 6) or (6, a). 

■ 

Proposition 10 

1. Q<n6 = r<. 

2. BgQ<. 

3. Q< ^BifW has at least three elements. 
4- Q< cBifW has one or two elements. 

Proof: 

1. Suppose -<;e Q< n B. Since -<€ Q<, it is irreflex- 
ive, so since it is in B, it is in 7^ by Proposi- 
tion 6. Suppose, instead, -<&T^. By Proposi- 
tion 6, -<e B. Let -<€ T be a relation such that 
-< is its strict vcirsion. (Obviously such a rela- 
tion must exist.) From Proposition 9, Q, so 

Q<. Thus, Q<r)B. 

2. The fully connected relation over W is in B, but 
not asymmetric and, so, not in Q<. 

3. Suppose a and b are distinct elements of W. If W 
has at least three elements, the relation {(a, b)} is 
not modular and, thus, not in B, yet it is the strict 
version of the relation W x W \ {(6, a)} which is 
total and quasi-transitive (since {(a, b)} is transi- 
tive). 

4. Suppose W has one element. Then B contains 
both possible relations over W, whereas Q< con- 
tains only the empty relation over W. 

Suppose W has two elements a and b. Then B con- 
tains empty relation, the fully connected relation. 



and all eight of the remaining relations which con- 
tain either (a, 6) or (6, a), but not both. Q<, on 
the other hand, only contains the three irreflexive 
relations. 



Proposition 11 If S C S, then Un{S) is modular but 
not necessarily transitive. 

Proof: Let ~<=Un{S). Suppose x,y,z&yV and 
X ~<y. Then there is some s £ S such that x <^ y. 

By assumption, is modular, so x z or z <^ y. 
By the definition of Un{S), x ~< z oi z ^ y, so ~< is 
modular. 

Suppose a,6, cSW and 5={si,S2} such that 
{{a,b),{a,c)} and <'^= {{b,a),{c,a)}. Un{S) 
is not transitive. ■ 

Proposition 12 If S C S, then AGRUn{S) e B. 

Proof: The transitive closure of any relation is tran- 
sitive. Since Un{S) is modular, the transitive closure 
of Un{S) is also modular by Proposition 1. ■ 

Proposition 13 // 5 C 5 and □s is a total order, 
then AGRRf{S) G B. 

Proof: We first prove that AGRRf{S) is modular. 
Note that the proof does not depend on ^s: being a 
total order. 

Lemma 13.1 If S C S, then AGRRf{S) is modular. 

Proof: Let AGRRf{S). Suppose x,y,z(^W 
and X ^ y. Then there exists s Cz S such that a; <" y 
and, for every s' ^ S, s' □ s implies x y and 
y -ft^ X. Consider a source s' such that x z, 
z a;, y z, or z y and, for every s" £ 
s" □ s' implies x z, z ft^ x, y fi" z, and z ft^ y. 
We know such a source exists since, by modularity, 
a; z or z <" y. Furthermore, since s' is a maximal 
rank such source, s' □ s. We consider the four cases 
and show that, in each, either x -< z ov z -< y: 

Case 1: x <^ z. Since, for every s" € S, s" □ s' im- 
plies X ■^''^ z and z ^zi* x. x ~< z. 

Case 2: z x. By modularity, z <" y or y <^ x. 

Suppose z y. Since, for every s" e S, s" □ s' im- 
plies z ^zi* y and y z, z < y. 

Suppose y <^ x. Then s □ s', so s = s'. Since x <''' y, 
a; z or <^ y by modularity. Thus, substituting 
s for s' above, either a; z and, for every s" G S, 
s" □ s implies x <fi^ z and z <jtf x so that a; -< z, or 



z <" y and, for every s" S S, s" □ s implies z y 
and y z so that z ~< y. 

Case 3: y z. By modularity, y a; or a; z 
which we have already considered in cases 2 and 1, 
respectively. 

Case 4: z y. We have already considered this in 
case 2. ■ 

It remains to show that -< is transitive. Suppose x ~< y 
and y < z. Then there exists si G 5 such that x y 
and, for every s'l € S, s'l □ s implies x 5^*1 y and 
y 5^^! .T, and there exists S2 G S' such that y z and, 
for every 53 G 5, S2 ^ s implies y ^zi'^s z and z 52^*2 y. 
Suppose si □ S2 (the case S2 □ si is similar). Then 
y ^zi'*^ z and z ^zj*^ y. By modularity, since x y and 
z fi*! y, X z. Let s' G 5 and s' □ Si. Then x 5^"' y 
and y a;. And, since si □ s' □ S2, so y -ft^ z 
and z y. By modularity, x z and z a;. 
Therefore, a; ^ z. ■ 

Proposition 14 //S' C 5, </ien AGR*{S) G S. 

Proof: By Proposition 12, -^^G for every 
r G ranks{S). For convenience, we assume the ex- 
istence of a "virtual" source Sr corresponding to 
each -<r- Precisely, for each r G ranks(S), assume 
there exists a source Sr & S such that <*'''=^r and 
rank{sr) = r, and let S' be the set of these sources. 
Then, 

AGR*{S) 

_ j ( ^ 3r G 7^. a; <r yA 1 
<^[x,y): (Vr' > r G ranfcs(5). a; y) J 

( 3s G S'. a; <^ yA 1 

= (vs'^seS'.x^^'y) j 

= AGRRfiS'). 

Since there is one source in S' per rank r, and since 
> is a total order over TZ, 3s' is a total order. The 
result follows from Proposition 13. ■ 

Proposition 15 If S C S, then AGR{S) G B. 

Proof: By Lemma 13.1, AGRRf{S) is modular. 
AGRRf{S)^ is obviously transitive, and, by Propo- 
sition 1, it is modular as well. ■ 

Proposition 16 Suppose S C S. 

1- If^s is fully connected, AGR{S) = AGRUn{S). 
2. If^s is a total order, AGR{S) = AGRRf{S). 
Proof: 



1. Suppose ^5 is fully connected. Then the 
second half of the definition of AGRRf 
is vacuously true so that AGRRf (S) sim- 
plifies to {(a;,y) : 3s G S*. x y}. But 
this is exactly Usgs Un{S), so 
AGR{S) = AGRRf {S)+ = Un{S)+ = AGRUn{S) 

2. Suppose □s is an total order. By Propo- 
sition 13, AGRRf (S) is transitive, so 
AGR{S) = AGRRf {S)+ = AGRRf {S). 



Proposition 17 Let S* = {si, . . . , s„} C 5 and 
AGRfi<^\. ..,<«-)= AGR{S). AGRj satisfi.es (the 
modified versions of) restricted range, unrestricted 
domain, Pareto principle, IIA, and nan- dictatorship. 

Proof: Let -<= AGRf{<^\ . . . , K^'^). Then 
^=AGR{S). 

Restricted range: AGRf satisfies restricted range 
by Proposition 15. 

Unrestricted domain: AGRf satisfies unrestricted 
domain by Definition 7. 

Pareto principle: Suppose x y for all s^. In par- 
ticular, x <^ y where s is a maximal rank source of S. 
Since s is maximal, it is vacuously true that for every 

s' □ s G S*, X -^^ y and y fj" x. Therefore, x -< y, so 
AGRf satisfies the Pareto principle. 

IIA: Let S' = {s[, . . . , s^}. First note that AGRRf 
satisfies IIA: 

Lemma 17.1 Suppose S — {si, . . . , s„} C S, 

S' = {s[,...,s'„} CS, Si = s'^ for all i, 
AGRRf {S), and AGRRf (S'). If for 
a;, y G W, x y iff x y for all i, then x y iff 
X y. 

Proof: Suppose Sj = s^, and x y iff a; y, for 
all i. Then x y iS x y since Definition 13 only 
relies on the relative ranking of the sources and the 
relations between x and y in their belief states to de- 
termine the relation between x and y in the aggregated 
state. ■ 

Thus, IIA can only be disobeyed when the closure step 
of AGR introduces new opinions. (Note that IIA is 
satisfied when there are no sources of equal rank since, 
by Proposition 16, the closure step introduces no new 
opinions under these conditions.) However, these new 
opinions are only added between worlds already in- 
volved in a conflict, as the following two lemmas show: 



Lemma 17.2 Suppose S QS and AGRRf{S). 
For every integer n>2, if x,y G W, x 7^* y, there ex- 
ist xq,. . . ,Xn € W such that x = xq -<*■■■-<* Xn = y, 
and n is the smallest integer such that this is true, then 

Xn ' ' ' '^'0 • 

Proof: Suppose x, ySW, x y^^, y, and there ex- 
ist Xq, . . . , a;„ € W such that a; = ^* • • • ^* a;„ = y, 
and n is the smallest integer such that this is true. 
Consider any triple 0:4-1,2:^, Xi+i, where 1 < z < n — 1. 
First, Xi-i 7^* Xi-^-l, otherwise there would be a chain 
of shorter length than n between x and y. Now, since 
Xi-i -<* Xi, there exists si G 5 such that Xi-i 
and, for all s' □ Si G S, Xi-i Xi. Similarly, 
there exists S2 G S such that Xi <''^ Xj+i and, for all 
s' □ S2 € S, Xi Xi+i. Thus, all sources with higher 
rank than max(si, S2) are agnostic with respect to 
and Xi+i. 

Suppose Si □ S2- Then Xi Xi+i so, by transitivity, 
Xi-i a^i+i. But then Xi-i ^* ajj+i, a contradic- 
tion. Similarly, we derive a contradiction if S2 C Si. 
Thus, si = S2. 

Now, since Xi-i 7^* aij+i and all sources with rank 
higher than si and S2 are agnostic with respect to Xi-i 
and Xi+i, Xi-i -ft^^ Xi+i. By modularity, Xi+i Xj. 
Since Si = S2, and all the sources with higher rank 
than S2 are agnostic with respect to Xi and Xi+i, 
Xi+i Xi. Similarly, Xi Xi-i, so ^« Xi-i. 
Since i was chosen arbitrarily between 1 and n — 1, 
Xn -<*■■■-<* Xq. And, in fact, all the opinions be- 
tween these worlds originate from sources of the same 
rank. ■ 

Lemma 17.3 Suppose S CS, AGRRf{S), 
-<= AGR{S), and x y^^y for x,y £ W. If x <y, then 
xooy. 

Proof: Suppose x -^^y. If x ^ y, then there exist 
xo, . . . , x„ such that x = xq -<*•••-<* x„ = y and n is 
the smallest positive integer such that this is true. 
Then, by Lemma 17.2, y = Xn Xq = x, so 

y -<x and xooy. ■ 

Now, suppose x,y G W, x y iff x <®» y for all i, 
x(^y, and xcjh'y. We show that x ~< y implies x -<' y 
(the other direction is identical). Suppose x ^ y. 
Let AGRRfiS) and AGRRf{S'). Since 
X9fc/y, X y by Lemma 17.3. But then x -<'^y by 
Lemma 17.1, so x ^' y. 
(END OF IIA SUB-PROOF) 

Non-dictatorship: Suppose is fully connected 
and suppose x y and y -ft^' x. Let Sj be such that 
y X. Then x -< y and y -< x, so Sj is not a dictator. 



(END OF PROPOSITION 17 PROOF) 

Proposition 18 Let A and S he as in Definition 21, 
-<^^ , agent Ai 's induced belief state, and ^g, fully con- 
nected. If A = @ (A), then (Ua e.A ^'^■)^ is A's in- 
duced belief state. 

Proof: We will use the following lemma: 

Lemma 18.1 If II is a set of relations over an arbi- 
trary finite set fl, then 

(m ) ) 

where <+ is the transitive closure of <. 

Proof: Let ^=(U<en<+)^, (U<en <)^, 

and a, 6 G f2. Suppose a <b. Then there exist 
<0j • • • , <n-i€ n and Wo, . . . ,Wn € such that 

a = Wo<o ■■■ <n-l Wn=b 

Thus, there exist xqo, • . • , xomo) • • •) 

X{n-1)0, ■ ■ ■,X(n-l)mn-i ^ SUCh that 

a = Wo = Xoo <0 • • • <0 Xomo = Wl = Xio ■ ■ ■ 

= Wn-1 = X(„_i)o <n-l • • ■ <ra-l X(n-l)mn-i 
= Wn = b 

SO a <' b. 

Now suppose a -<' b. Then there exist 

<0j • • • , <n-i€ n and Wo, . . . ,Wn G O such that 

a = Wo <0 • • • <n-l Wn = b 

Obviously, this implies that 

a = Wo<o ■■■ <n-l Wn=b 

which implies that 

a = wo<t ■■■ <t Wn = b 

where <*= ^U<en so a < 6. ■ 

Now, let -< be the belief state induced by @ {A) . Then 
-<= AGR{S). By Proposition 16, -<= AGRUn{S), so 

~< = Un{S)+ 




By the lemma, 




Proposition 19 Let A he an agent informed by a set 
of sources S C S and with pedigreed belief state {~<,l)- 
Then 

X^rV 

iff 

3s € S. X <^ y Ar = rank{s)A 
(ys' □ s G 5. a; w^' 2/) . 

Proof: Suppose x y. Then x -< y and 

l((x,y))—r. By Definitions 13 and 22, there 
exists s G S such that x <" y and for every s' □ s G 5, 
X y. In particular, if a; y for some s' G S, then 
s □ s', so rank{s) > rank{s'). Thus, 

r = l{{x,y)) 

= ma.x{rank{s') : x <^ y,s' & S} 
= rank{s). 

Now suppose there exists s € S such that x <^ y, 

r = rank(s), and, for every s' H s E S , x y. Then 
X ^ y. Moreover, since for every s' G 5, a; y im- 
plies s 3 s' which implies rank{s) > rank{s'), 

l{{x,y)) = max{ranfc(s') : a; y,s'€S} 
= rank{s) 
= r. 

Therefore, x y. ■ 

Proposition 20 Let A and S be as in Definition 21, 
^s, 0, total pre- order, and A = @ (A). Lf 

1. -< is the relation 

3Ai E A,r en. X yA | 
{x, y) ■ ^j^, &Ay >r&Tl. X ^f/ j 

over W, 



2. I -.^^U such that 

l{{x, y)) = max{r : x y, Ai G A}, and 

then (-<, /) is A 's pedigreed belief state. 

Proof: Let -<'= AGRRf{S) and /' 11 such 

that l'{{x,y)) = max{rank{s) : x <^ y,s G S}. It suf- 
fices to show that -<=^' and / = I'. 

Suppose X -<y. We show that x -<' y, i.e., there ex- 
ists s G S such that x <^ y and, for every s' □ s G 5, 
X -^^ y and y -^^ x, and that l'{{x,y)) = l{{x,y)). 
Since x ~<y, there exists Ai and r such that x -<^^ y 
and, for every Aj G A and r' > r G 71, x y^f,^ y and 
y X. Since x -<^^ y, there exists s € Si such 
that X y, rank(s) = r, and, for every si □ s G Si, 
X -^"^ y and y -^"^ x. Si C S, so there exists s G 5 
such that X y. Now suppose s' is a maximal rank 
source of S with .x <^ y or j/ <* .t. Such an ,s' exists 
since x y. Since □ is a total pre-order, it suffices 
to show that s 3 s' . Suppose s' G Sj. Since Sj C 5, 
s' is also a maximal rank source of Sj with a; y 
or w a;, so a: ^ ^ , , ,^ y ov y < \ , x. But since 

^ ' rank{s') ^ ^ rank[s') 

X y, r = rank{s) > rank{s'), so s □ s' . Further- 
more, l'{{x,y)) = rank{s) = r = l{{x,y)). 

Now suppose X -<' y. We show that x -< y, i.e., there 

exists Ai and r such that x -<^' y and, for every 
Aj G A and r' > r GTZ, x ^^i' y and j/ y^f/ x, and 
that l{{x,y)) = l'{{x,y)). Since a; -<' y, there exists 
s € S such that .x j/ and, for every s' Zl s £ S, 
X y and y a;. Suppose s G Si. Since iSj C S, 
it is also the case that for every s' ^ s € Si, x ft^ y 
and y x, so x -<f*nk{^-^ y- Now, let Aj and r' be 

A ■ A 

such that X -<^i' y or y .t. It suffices to show that 
rank{s) > r'. By Proposition 19, there exists s' G Sj 
such that a; y or y <^ x and rank{s') = r'. But 
then s □ s' , so rank{s) > rank{s') = r' . Furthermore, 
K{x, y)) = rank{s) = l'{{x, y)). ■ 

Proposition 21 If A and S are as in Definition 21, 
□ s is a total order, and {^,1) is the pedigreed belief 
state of® (A), then ^+=-<. 

Proof: Since 3 s is a total order, 
AGRiS) = AGRRfiS) by Proposition 16. Thus, 
~<= AGRRfiS) = AGR{S) = AGRRf{S)+ =-<+. ■ 



B Notation key 



Cl: arbitrary finite set 

a,b,c, . . .: specific elements of a set 

X, y,z,. . .: arbitrary elements of a set 

A, B,C, . . .: specific subsets of a set 

X.Y, Z, . . .: arbitrary subsets of a set 

H: arbitrary set of relations 

<: arbitrary relation 

C{X, <): choice set of W wrt < 

W: finite set of possible worlds/alternatives 
w, W: element, subset of W, respectively 

B: set of generalized belief states (modular, transitive 
relations) 

-<: element of B, strict likelihood/preference 
■<: weak likelihood/preference 
~: equal likelihood, agnosticism/indifference 
oo: conflict 

T: set of total pre-ordcrs 

: strict versions of total pre-orders 
Q: set of total, quasi-transitive relations 
Q< : strict versions of total, quasi-transitive relations 

S: set of sources 

s, S: element, subset of <S, respectively 

belief state of source s 
«: source agnosticism 
CXI: source conflict 
TZ: set of ranks 
r: element of TZ 

3, 3s: credibility ordering over >S, S CS, respectively 

A: set of agents 
A: element of A 
-<^: A's induced belief state 
pedigreed belief state 
I: labeling function of a pedigreed belief state 
-<^: restriction of A's pedigreed belief state to rank r 



